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COST MODELING TECHNIQUES FOR DESIGN MATURITY 
E. W. Rvihland 
Jet Propulsion Laboratory 

MR. RUHLAND: It is a pleasure to be here and not have some- 
thing controversial to talk about, because I didn't have anything 
to do with generating any numbers here. So I'm just talking phi- 
losophically . 

The first point I'd like to make is I'm in a very delightful 
position at JPL: I'm never right and I'm never wrong, because 

before a project comes in, my estimate is always too high; once 
it's through the door, it's too low. But, then, I'm never wrong 
because they never do the project that was estimated. 

As a matter of information. Figure 10-3 shows some things 
that we have available at JPL. I'd like to say that they're only 
available to the government. They are not available to contrac- 
tors; maybe they are lucky. 

The first one is a model on re-entry heatshields, aerody- 
namic decelerators , and the integration problems particular to 
that. The model only works with the second model shown on the 
figure and I would like to point out the date, 1970, which makes 
it old. I would also like to point out the development of this 
model was funded by Dan Herman, as a matter of fact, in one of 
his studies, and we did a grand total of two man-months of effort 
on it and now use it as a guide for trade-offs. 

In general, I want to talk about what drives subsystem costs 
and how maturity affects it (c.f. Figure 10-4). Basically, given 
a technology base, the cost is driven by the number of interfacing 
svibsystems. The more interfacing subsystems you have, the higher 
the price tends to go. Subsystem costs are also driven by; the 
design and software maturity and I am using "maturity" the way 
some people might use inheritance; the test effort; and changes in 
the interfacing subsystems. And, cost avoidance items are hardware 
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AVAILABLE REPORTS AT JPL 

O RE-ENTRY SUBSYSTEM COSTS (REF. 1) INCLUDE 
HEAT SHIELD 

AERODYNAMIC DECELERATOR 
VEHICLE INTEGRATION (R/V TO S/C) 

O TO BE USED IN CONJUNCTION WITH BASIC COST MODEL 
(REF. 2) 

REFERENCE 1. F. E. Hoffman, "Cost Prediction Model for Unmanned 

Space Exploration Missions -- Entry Structure Cost 
Parameters, " R-1434, dated April 24, 1970. 

2. F. E. Hoffman, et al. , Cost Prediction Model for 

Unmanned Space Exploration Missions, PRC R-1298, 
dated December 15, I9()9 
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availability, software availability, and support equipment avail- 
ability. These are the true inheritors. Finally, the level of 
documentation has a cost impact. For example, we can track in 
Viking some of the effects of the level of documentation. All 
these things have to be considered if we are ever going to have 
a low-cost approach in doing business. 

At the laboratory, we have developed internal to the cost- 
estimating people - we tried to hide this from the JPL'ers so I 
hope they are not taking notes - a maturity index as shown on 
Figure 10-5. The basic concept of the maturity index is to bracket 
the level of the subsystem, its status. It begins at the highest 
index represented by existing, qualified hardware, or that which is 
in active production, i.e. you are going to do the same thing the 
same way. It proceeds then, the next step down, to either a modi- 
fication of the hardware or which you have to qualify because there 
is a new environment that's different in some way, or you have to 
replicate the qualified design. We find when we analyze the cost 
data that if you can't get onto an existing line you can't achieve 
the inheritance that you would like. 

Next, down the maturity index scale is extending the sub- 
system capability using qualified piece parts. For example, mak- 
ing a bigger computer out of the same piece parts. Or either of 
the items from the index above, where you have to qualify the 
modification or extend the time. As you spread out in time, you 
pick up more cost and this starts getting subjective. There is no 
question; trying to cost maturity or to take into account maturity 
requires subjectivity, it requires a great deal of understanding 
of what you are trying to do, and it takes a great deal of open- 
channel communication with the technical and project people to 
keep you informed of the actions and status. 

The lower end of the maturity index is zero, where we have 
never done the subsystem before, new technology is required and 
we bracket to that level. 
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Figure 10-6 shows the effect of maturity on cost. One of 
the major points is that design changes flow toward the less 
mature subsystem. This is not really a continuous curve, it is 
a continuous curve for purposes of modeling. Actually, there are 
great pressures not to change the design if you have a high ma- 
turity level. There are great pressures at the very low levels 
of maturity (and high cost amplifier levels) to force the design 
toward higher maturity indices, so you .jtend to have a cost ampli- 
fier that goes up from the lower right and it sort of settles 
toward the middle. The most important point is that the inflec- 

I 

tion point tends to move with the changes in the interfacing sub- 
systems. For example, if you have an existing computer but you 
are changing everything around it you are going to have to spend 
more money on the computer anyway. 


This is the basic approach, philosophically. We have de- 
veloped the CER's on this and we have tested it and it seems to 
be working fairly well. 


The second most important cost driver that we tried to 
model is the test effort as shown on Figure 10-7. The test 
program is structured by the mission and the design complexity, 
the mission and program risk avoidance (or acceptance) and by 
design maturity of the system and subsystems. Someone, at some 
level, has to say, "I will accept less testing and more risk to 
save cost." I can verify for example that, with time, at JPL we 
have been willing to do less testing on the Mariner machine be- 
cause we understand it better. The designers understand it better. 
The general structure of a test program tends to be directed 
towards detecting design and fabrication defects at each level. 

You test at the vehicle level, the system, subsystem, assembly, 
and at the piece part level with a minimum of some kind of screen- 
ing. And accepting, or neglecting testing at any one of those 
levels is a major cost driver. It is a programmatic decision, a 
risk. 
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We have also developed a CER on which we have been working 
with regard to the effect of test level on cost. I show a sani- 
tized picture of this on Figure 10-8. Basically, it results from 
the following things: As the number of test levels increases, the 

number of interfaces, the number of tests and support equipment 
increases. That is a direct cost driver. The impact is directly 
dependent on design maturity and, in general, it tends to look 
like the two curves on the figure. As the number of tests and 
test levels increase, the cost amplifier goes up exponentially 
because you pick up increasing integration costs, increasing sup- 
port equipment costs, etc. Design maturity, however, can lower 
the cost amplifier as shown. But also you must not forget that 
with maturity the number of tests also comes down. You can't 
forget that this tendency to push down is directly affected by 
constraints and risks in management. To lower the cost for ex- 
ample by removing subsystem people from the project before you 
complete system testing, you are accepting a risk. If you don't 
want to accept the risk, you have got to expend the money neces- 
sary to continue to support the subsystem people. 

That is really all I had to go over in a general presentation 
I didn't realize this was to be an open meeting and I had prepared 
a presentation containing numbers that I am unable to release to 
an open session. 

MR. VOJVODICH; Thank you very much. Bill. Do we have any 
questions? 

MR. CASANI; Yes, Let's see. Bill, you made a comment on the 
reduced level of testing you have experienced at JPL. Could you 
be more specific? We have looked at the series of Mariner programs 
Is the percentage of total dollars that is being spent on testing 
decreasing? 

MR. RUHLAND: The percentage has not been decreasing be- 

cause as the design goes down, the testing goes down somewhat in 
parallel. So you might say that it's tending to stay a constant 
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Figure 10-8 



percentage of a reduced number. Because of maturity, you have an 
interaction in the cost of the design and the testing: You do 

less design and you do less testing. So the testing costs have 
gone down on a noinnalized basis, but they tend to stay the same 
percent because the design cost comes down. 

MR. SEIFF: I found your presentation to be really a dream 

because it looks like you are trying to quantify something that 
has been generally something of a black art, you know, sort of a 
guessing game. I was just wondering how far this quantification 
goes in terms of - take a new program like this one that is being 
discussed here. Do you actually proceed from a set of charts? 

The ones you showed us were qualitative; they had no numbers on 
the axes. 


MR. RUHLAND; I painted the numbers off. 

MR. SEIFF; You take a set of charts and apply them, sub- 
system by subsystem, and end up with a final estimate. Do you 
then try to bring judgmental factors in at that point or how do 
you actually do this; and what has been your experience in pre- 
dicting the accuracy of the end result? 

MR. RUHLAND: We try to push the subjectivity to the farthest 

front point that we can, and we quantify all the operations there- 
after. The subjectivity comes from a dialogue with the technical 
people, trying to understand what they mean when they say they are 
inheriting this or they are expanding that, and to turn it into 
an input factor. But we have been doing this for six years now. 

We started trying to track inheritance six years ago and we have 
been learning. We did terribly for a while and we finally got down 
to something that I think is working right now. A priori it tracks 
about thirty missions, when you use the maturity factor, within 
about a three-percent band. On a new project it's probably track- 
ing twenty percent but how much of that is the model and how much 
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of that is understanding of the project? What I said in the be- 
ginning wasn't a joke; it's literally true. The project I esti- 
mate with a model, before there is a project office is never done. 
When you get a project office and they see the problems they've 
got and they really try to buy the hardware that they can't get 
now, the project is restructured. So, literally, they never do 
the project that is estimated by the model before the fact. Now, 
at JPL I track every project until completed. I continuously 
re-estimate and I can see it coming in. They change and we con- 
verge. If you don't know the project, you can't get the cost. 

MR. HERMAN: Just one question: On MJS what was the vari- 
ance between 's estimate and the estimate as submitted by 

your model? 

MR. RUHLAND: We came within plus or minus five percent on 

the mean. I don't remember precisely. I think it might have 
been plus or minus four percent, so that is an eight percent band 
width . 


MR. HERMAN: Is the project that you modeled the project 

that Boyster and Meyers are implementing now? 

MR. RUHLAND: Pretty much. I have done a couple more model 

runs since. I do a minimum of one model run on every project a 
year, and the last model run I did, I talked to Hickock and the 
people and we got the deltas and changes, and we went through there. 
There were not many changes in the assumptions to make a model run, 
number one; they were very close. The numbers still tracked about 
the same way . 
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